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1.  INTRODUCTION 


This  Final  Technical  Report  reviews  the  results  of  our  support  for  the  USAF  Electronic 
Systems  Division  (ESD)  in  conducting  the  Technology  Validation  Experiment  (TVE)  over 
the  DARPA  Wideband  Satellite  Network.  The  ESD  site  in  Bedford,  MA  contained 
computers  running  the  Cronus  Distributed  Operating  System,  as  did  sites  at  the  Rome  Air 
Development  Center  (RADC)  in  Rome,  NY  and  BBN  Systems  and  Technologies 
Corporation  in  Cambridge,  MA.  The  goal  of  the  TVE  is  to  demonstrate  that  a  large 
computer-based  simulation  can  be  run  in  a  distributed  computation  environment  provided 
by  Cronus  Distributed  Operating  System  "clusters"  located  at  the  ESD,  RADC,  and  BBN 
testbeds.  Relatively  high-bandwidth  data  paths  are  required  for  the  support  of  the 
simulation’s  wide  area  intercluster  communications. 

This  contract  provided  for  the  support  of  the  TVE  at  ESD.  Major  task  areas  included  the 
provision  of  Wideband  connectivity,  on-going  operational  consulting  and  repair, 
analysis  of  data  communication  media  and  protocol  performance,  and  monitoring  and 
control  services. 
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2.  OVERVIEW  OF  PROBLEM  AND  OBJECTIVES 
2.1.  COMMUNICATION  EQUIPMENT 

The  major  task  was  to  supply  network  connectivity  among  the  Cronus  machines  at  the  three 
sites.  Cronus  machines  on  the  Local  Area  Networks  (LANs)  at  BBN  already  had 
connectivity  with  the  Cronus  machines  at  RADC's  LAN  via  the  existing  Butterfly  Internet 
Gateways  and  the  DARPA  Wideband  Satellite  Network.  To  add  network  connectivity  with 
ESD  Cronus  hosts,  a  Butterfly  Internet  Gateway  was  procured  for  the  ESD  site.  With  a 
distance  of  several  miles  between  the  ESD  site  at  Mitre  Corporation  in  Bedford, 
Massachusetts  and  the  nearest  Wideband  Earth  Station  at  Lincoln  Laboratory,  a  T1  (1.5 
Mbits/sec)  circuit  was  ordered  to  complete  the  communications  paths. 

Several  months  into  the  contract,  the  ESD  site  needed  to  relocate.  ESD  plans  required 
moving  the  TVE  SDI  staff  and  equipment  from  the  Mitre-B  building  in  Bedford,  MA  to  the 
Mitre-T  building  in  Lexington,  a  distance  of  several  miles,  and  actually  closer  to  the 
Lincoln  Laboratory  Wideband  site.  This  led  to  cancelling  the  T1  line  and  ordering  a 
replacement  line  to  the  new  building.  The  Butterfly  Internet  Gateway  was  moved,  and  re¬ 
installed  on  the  appropriate  ethemet  in  the  Mitre-T  building. 

An  independent  task  for  network  connections  between  the  Jet  Propulsion  Laboratory  (JPL) 

and  ESD  was  added  later  in  the  contract  period.  The  aim  of  the  additional  connectivity 

was  to  support  SDI  simulation  work  being  conducted  joindy  between  ESD  and 

JPL.  A  high-bandwidth  path  between  a  JPL  Hypercube  multiprocessor 

and  ESD  Sun  and  Silicon  Graphics  Iris  workstations  was  required.  The 

ESD  end  required  another  ethemet  interface  card  in  the  existing  Butterfly  Internet  Gateway, 

plus  the  network  cabling.  The  JPL  end  required  another  Butterfly  Internet  Gateway  with 

local  ethemet  interface  and  cabling,  plus  an  additional  T1  circuit  between  JPL  at  Pasadena, 

CA  and  the  nearest  Wideband  node  at  Information  Sciences  Institute  (ISI)  in  Marina  del 

Rey. 
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BBN  took  responsibility  for  coordinating  installation,  maintenance,  and  operation  of  the 
communication  equipment  supporting  the  Wideband  usage  by  ESD. 

2.2  PERFORMANCE  ANALYSIS 

With  the  Cronus  TVE  simulation,  the  performance  of  the  network  was  critical  to  the 
elapsed  time  of  an  experiment.  The  project  provided  an  opportunity  to  compare  a 
distributed  application  in  the  LAN  and  WAN  environments.  We  studied  both  the  vendor- 
supplied  network  software  and  the  underlying  Cronus  use  of  the  available  bandwidth. 

With  data  gained  from  the  studies,  a  more  flexible  set  of  Cronus  data-passing  calls  has  been 
designed  and  is  being  implemented. 

2.3  NETWORK  MONITORING 

Three  monitoring  changes  evolved  over  the  length  of  the  contract.  First,  the  Wideband 
monitoring  and  control  software  was  ported  from  the  C/70  host  run  by  the  NOC  to  a  Sun 
workstation  environment,  which  is  more  commonly  available.  Next,  a  Cronus  application 
was  developed  which  would  give  even  a  naive  user  a  graphical  picture  of  the 
communication  components  (BSATs  and  Gateways)  and  the  Cronus  hosts.  Finally,  a  new 
Wideband  Network  Monitor  was  implemented  as  a  prototype  of  a  future  graphical  tool.  It 
was  built  upon  existing  Automated  Network  Management  (ANM)  tools.  The  Cronus 
TVE  work  involved  multiple  network  technologies,  host  system  software,  and  host 
application  programs.  Monitoring  and  control  in  the  resulting  environment  demands  a 
clear  display  and  an  intuitive  command  structure. 
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3.  METHODOLOGY  AND  TECHNICAL  RESULTS 


The  initial  ESD  TVE  experimentation  began  with  a  new  high-speed  wide-area  connection 
established  between  an  ESD  testbed  facility  in  Bedford,  MA  and  testbeds  at  the  Rome  Air 
Development  Center  (RADC)  in  Rome,  NY  and  at  BBN  Systems  and  Technologies 
Corporation  (BBN)  in  Cambridge,  MA.  It  is  provided  via  a  BBN  Butterfly  Internet 
Gateway  that  was  installed  at  the  ESD  testbed,  interfacing  an  Ethernet  LAN  at  ESD  to  the 
DARPA  Wideband  Network  via  a  terrestrial  T1  circuit.  The  T1  circuit  terminates  at  the 
gateway  and  at  the  Wideband  Network  packet  switch  node  (known  as  a  BSAT)  at  MIT 
Lincoln  Laboratory  in  Lexington,  MA.  The  Cronus/TVE  cluster  connectivity  is  shown  in 
Figure  1. 


DARPA 

Wideband 

Network 


Figure  1.  Cronus/TVE  Cluster  Connectivity 
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A  number  of  engineering  issues  were  addressed  in  the  installation  and  initial  use  of  the  ESD 
Wideband  Network  connection.  One  of  these  was  the  determination  of  the  most  suitable 
link  technology  for  the  ESD-Lincoln  Tl-rate  connection.  A  leased  terrestrial  T1  circuit  was 
chosen,  installed,  and  tested  for  this  purpose  in  favor  of  the  use  of  microwave  link 
technology  after  a  site  survey  revealed  the  lack  of  a  clear  line  of  sight  between  the  ESD  and 
Lincoln  Laboratory  facilities. 

Other  engineering  issues  related  to  both  the  provision  of  an  Ethernet  LAN  for  the  ESD 
Cronus/TVE  hosts  and  to  the  provision  of  ARPANET  connectivity  to  the  ESD  site  as  a 
redundant  backup  path  for  enhanced  TVE  survivability.  Since  the  ESD  facility  where  the 
TVE  hosts  and  Butterfly  Gateway  were  to  be  located  was  found  to  be  already  equipped 
with  an  available  Ethernet  LAN  (the  MitreNet),  a  decision  was  made  to  home  the  hosts  and 
gateway  on  that  LAN  rather  than  install  a  new  dedicated  Ethernet.  The  MitreNet  connection 
also  provides  the  advantage  of  backup  connectivity  to  the  RADC  and  BBN  sites  via  a  path 
other  than  the  ESD  Butterfly  Gateway.  Such  backup  connections  are  routed  through  a 
VAX  host  acting  as  an  Internet  gateway  on  the  MitreNet  to  the  DDN  (Mil net)  and  from 
there  to  the  ARPANET  via  a  DDN  mailbridge  gateway.  (Both  the  RADC  and  BBN  TVE 
testbeds  are  equipped  with  ARPANET  access  via  Butterfly  Internet  gateways.)  Given  the 
availability  of  this  backup  Internet  path  to  the  TVE  hosts  at  ESD,  a  direct  interface  to  the 
ARPANET  was  not  installed  on  the  ESD  Butterfly  Gateway  as  was  originally  planned. 

The  establishment  of  initial  TCP  connections  between  the  TVE  hosts  at  ESD  and  their 
counterparts  at  RADC  and  BBN  required  modifications  to  the  ESD  hosts'  routing  tables  to 
use  the  ESD  Butterfly  Gateway  as  the  favored  path  to  the  other  sites.  With  the  correct 
implementation  of  these  routing  table  changes,  ESD-RADC-BBN  connectivity  via  the 
Wideband  Network  became  routine.  In  general,  the  connections  established  via  the 
Wideband  Network  have  provided  performance  superior  to  connections  established  over 


5 


BBN  Systems  and  Technologies  Corporation 


ARPANET  paths.  This  is  due  to  the  high-speed  nature  of  the  Wideband  path  as  well  as  to 
congestion  frequently  encountered  within  the  ARPANET. 

Due  to  access  restrictions  enforced  between  the  ARPANET  and  Milnet  communities  of 
hosts,  the  Butterfly  and  VAX  Gateways  homed  on  the  MitreNet  do  not  exchange  any 
Internet  routing  information.  This  restriction  currently  results  in  a  requirement  for  manual 
ESD  host  routing  table  modifications  when  switching  to  the  VAX-provided  path  upon  a 
failure  in  some  part  of  the  Wideband  path.  Although  there  are  TCP/IP  hosts  capable  of 
automatic  changeover  capability,  the  standard  Sun  Unix  and  DEC  VMS  host  software  does 
not  do  this.  Adding  this  enhancement  is  desirable  for  an  operational  Wide-Area  Cronus 
cluster,  but  not  necessary  at  this  stage  of  use. 

An  opportunity  for  a  test  run  of  a  wide-area  distributed  TVE  simulation  supported  by  the 
Wideband  Network  was  provided  at  the  December  1987  RADC  Distributed  Systems 
Technology  Exchange  Meeting.  Although  this  test  run  was  generally  successful,  it  was 
limited  to  only  the  RADC  and  BBN  Cronus  clusters  due  to  problems  encountered  with 
satellite  channel  connectivity  at  the  Lincoln  Laboratory  Wideband  Network  site. 

As  the  use  of  the  distributed  Cronus  clusters  became  a  reality,  other  overall  ESD  plans 
impacted  the  three  site  experiments.  The  ESD  cluster  and  Butterfly  Gateway  were  located 
in  ESD  facilities  at  Mitre- B  Building  in  Bedford,  MA.  ESD  needed  to  relocate  the  Cronus 
cluster  to  different  facilities  a  number  of  miles  away  at  Mitre-T  Building  in  Lexington,  MA 
in  the  early  part  of  1988.  The  Butterfly  Gateway  and  the  ESD  termination  point  of  the 
ESD-Lincoln  T1  circuit  had  to  be  moved,  and  the  gateway’s  connectivity  to  Lincoln 
Laboratory  had  to  be  established  from  the  new  location,  in  order  to  resume  ESD's  wide- 
area  inter-cluster  connectivity  to  RADC  and  BBN  via  the  Wideband  Network. 

A  tool  was  developed  to  periodically  measure  and  record  the  end-to-end  throughput 
achieved  over  TCP  connections  made  between  a  Sun  workstation  host  belonging  to  the 
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BBN  Cronus/TVE  cluster  and  a  peer  Sun  host  in  the  cluster  at  RADC.  This  tool 
automatically  wakes  up  every  six  hours,  creates  the  TCP  connection,  reliably  transfers  a 
megabyte  of  data  over  that  connection,  measures  the  effective  throughput  for  the  data 
transfer  (excluding  connection  setup  time  where  possible),  and  electronically  mails  the 
results  to  interested  parties.  A  capability  was  developed  to  graphically  display  the  resulting 
network  throughput  performance  data  sorted  by  either  data  transfer  durations  or  by  the  time 
of  day  of  the  tests.  This  tool  provides  a  "user"  view  of  the  wide-area  communication  layer 
performance  that  can  be  expected  by  the  Cronus  TVE  application. 

Performance  data  was  continually  collected  using  the  measurement  tool  over  an  extended 
period.  Although  the  measurement  tool  cannot  directly  determine  the  Internet  path 
taken  during  a  given  data  transfer  (i.e.,  Wideband  Network  vs.  ARPANET  based  paths),  it 
does  provide  information  on  the  initial  Internet  gateway  used  for  the  connection,  from 
which  the  path  may  be  inferred.  The  throughput  performance  of  connections  via  the 
Wideband  Network  has  been  found  to  be  relatively  constant  at  about  16  Kbit/s,  the 
maximum  throughput  to  be  expected  given  the  standard  SUN  TCP  window  size  and  given 
the  Wideband  Network  path's  round  trip  delay. 

An  effort  was  initiated  during  the  contract  period  to  locate  and  correct  the  cause  of 
unusually  large  intersite  frequency  offsets  which  were  being  observed  on  the  Wideband 
Network's  satellite  channel;  an  overly  large  frequency  distribution  can  result  in 
degradations  in  satellite  link  performance  which  can  adversely  affect  TVE  performance. 
After  investigation,  the  source  of  the  problem  was  found  to  be  incorrect  adjustment  of  the 
synthesizers  attached  to  the  upconverters  in  the  Wideband  earth  stations.  BBN  worked 
with  Contel  ASC,  the  Wideband  earth  station  provider,  to  correct  this  problem  the 
Wideband  sites. 

Problems  encountered  at  the  Lincoln  Laboratory  Wideband  Network  site  which  had 
prevented  ESD’s  full  participation  in  previous  TVE  test  runs  were  corrected  in  January 
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1988.  A  bad  IF  cable  connection  within  the  Lincoln  earth  station,  a  likely  contributor  to  the 
site-'s  intermittent  satellite  channel  connectivity,  was  repaired.  In  addition,  a  number  of 
bugs  were  identified  and  corrected  in  the  BSAT's  processor  node  and  channel  I/O  device 
software.  Some  of  these  bugs  had  been  responsible  for  sporadic  BSAT  system-level 
restarts  that  were  causing  failures  in  TCP  connections  terminating  at  the 
ESD  TVE  hosts. 

On  February  17,  1988  the  Wideband  Network  and  Butterfly  Internet  Gateways  provided 
the  wide-area  connectivity  for  a  major  demonstration  of  the  TVE  distributed  simulation  that 
was  given  by  BBN  Systems  and  Technologies  Corporation  to  ESD.  This  successful 
demonstration  included  all  three  of  the  TVE  sites:  ESD,  RADC,  and  BBN. 

A  Lincoln  Laboratory  Butterfly  Internet  gateway  was  installed  in  March  1988,  directly 
connecting  the  Lincoln  BSAT  to  the  ARPANET,  and  creating  additional  backup  path  for 
the  provision  of  wide-area  TVE  connectivity  to  ESD.  This  path  provides  ARPANET 
access  in  cases  where  there  is  an  isolated  failure  in  the  Lincoln  Wideband  site's 
satellite  channel  subsystem  (and  all  other  communication  subsystems  remain  operational). 

As  mentioned  above,  automated  performance  measurements  show  the  throughput  of 
intercluster  connections  routed  via  the  Wideband  Network  to  be  relatively  constant  at  about 
16  Kbit/s.  This  result  is  a  function  of  the  standard  TCP  window  size  of  4096  bytes  (32768 
bits)  used  by  the  Sun  workstations  and  the  approximately  2  second  end-to-end  round  trip 
delay  characteristic  of  the  Wideband  Network  path.  The  2  second  Wideband  path  round 
trip  delay  represents  the  sum  of  satellite  channel  propagation  delays  (two  1/4  second  "hops" 
in  each  direction  for  Wideband  Network  datagram  transmission  =>  ~1  second  of 
propagation  delays),  current  Wideband  distributed  scheduling  site  synchronization  delays 
(-1/3  second,  round  trip),  and  assorted  buffering,  queueing,  and  processing  delays  in  the 
Butterfly  gateways  and  BSATs.  The  round  trip  delay  dominates  the  network  transmission 
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rate  in  determining  the  maximum  achievable  TCP  connection  throughput  over  the 
Wideband  path  since  the  transmission  time  of  a  complete  4096-byte  window  of  data  is 
negligible  compared  to  the  round  trip  delay. 

Experiments  were  performed  to  determine  the  best  intercluster  Wideband  path  throughput 
that  could  be  achieved  between  Sun  workstations  via  modifications  to  standard  available 
versions  of  TCP.  The  experiments  resulted  in  an  increased  throughput  of  approximately  64 
Kbit/s.  This  result  was  achieved  via  integration  of  the  latest  Berkeley  TCP  release  into  the 
Sun  UNIX  kernel,  with  the  TCP's  window  size  optimized  (increased)  for  the  Wideband 
path.  Known  bugs  in  the  Sun  kernel  prevented  a  throughput  increase  beyond  64  Kbit/s; 
such  an  increase  would  require  an  experimental  improved  version  of  TCP  with  special 
options  and  better  flow  control  to  solve  the  high-speed/high-delay  network  problem. 
Improvements  along  such  lines  could  be  addressed  in  the  future  if  required  by  the  TVE 
application. 

New  firmware  was  installed  in  the  "Earth  Station  Interface"  (ESI)  satellite  burst 
modem/codec  units  that  provide  satellite  earth  station  connectivity  at  all  of  the  Wideband 
Network  sites,  including  Lincoln  Laboratory,  RADC,  and  BBN.  The  new  software 
provides  performance  improvement  in  ESI  processing  of  satellite  channel  bursts  via  the  use 
of  larger  buffers  and  more  available  memory  for  burst  data  storage.  The  intent  of  this 
change  was  a  decrease  in  the  frequency  of  satellite  channel  connectivity  and 
burst  data  lossage  in  the  ESI  under  conditions  of  high  channel  traffic  loads;  this  should 
improve  the  quality  of  wide-area  TVF,  connections. 

Due  to  facilities-related  problems  at  Mitre-T  Building,  the  installation  of  the  re-homed  T1 
circuit  providing  connectivity  from  T  Building  to  Lincoln  Laboratory  was  not  completed 
until  mid-June  1988.  When  the  installation  was  completed,  BBN  personnel  began  work  on 
re-integrating  the  ESD  Butterfly  Gateway  (moved  from  its  previous  Mitre-B  Building 
location)  with  the  Lincoln  Laboratory  BSAT  and  the  ESD  Cronus  cluster's  Ethernet  LAN. 
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Several  problems  in  the  operation  of  the  re-homed  T1  circuit  prevented  the  reestablishment 
of  stable  Wideband  connectivity  for  ESD  until  July. 

Two  efforts  were  initiated  in  July  1988  with  the  goal  of  improving  the  operation  and 
efficiency  of  the  Cronus  TVE  when  executing  in  a  multi-cluster  wide-area  network 
environment  such  as  is  currently  provided  by  the  Wideband  Network  and  the 
ARPANET/DDN.  The  goal  of  the  first  effort  was  to  make  Wideband  Network  monitoring 
data  available  within  the  Cronus  system.  This  capability  was  developed  by  porting  the 
Wideband  Network  monitoring  system  from  its  original  BBN  C/70  computer  base  to  the 
Sun  Workstation  environment,  and  by  implementing  a  Cronus  front-end  manager  and 
display  which  allows  Wideband  status  information  to  be  integrated  with  other  data  currently 
available  to  Cronus  users.  The  Wideband  monitoring  data  are  contained  in  a  status  file 
which  defines  the  primary  interface  between  the  Cronus  manager  and  the  "satellite  network 
poller"  component  of  the  Wideband  monitoring  software.  Operators  of  the  Cronus  system 
and  Cronus  application  processes  can  use  this  new  view  into  the  underlying  network  layer, 
enabling  them  to  optimize  inter-cluster  communication  decisions  based  on  the  information 
presented.  Figure  2  is  an  example  of  a  Cronus  display  integrating  both  Wideband  node  and 
Cronus  host  status  information.  The  standard  Wideband  network  monitoring  displays 
were  also  also  be  ported  to  the  Sun  environment  to  ensure  that  network  monitoring  data 
will  be  available  for  diagnostic  purposes  even  when  the  Cronus-based  monitoring 
capabilities  are  not. 
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Wideband 


Figure  2.  Cronus  Display  of  Wideband  Node  and  Host  Status 

The  Cronus  TVE  operates  by  passing  continuous  streams  of  data  between  distributed 
computing  resources.  While  this  offers  opportunities  for  concurrent  processing  through 
pipelined  control  strategies,  such  strategies  had  not  been  implemented.  This  is  of 
particular  significance  in  the  TVE’s  wide-area  networking  environment,  where  Cronus'  use 
of  TCP/IP  along  with  a  synchronous  invoke/reply  protocol  for  each  operation  results  in 
significant  delays  when  data  streams  pass  between  remote  sites.  The  second  effort  initiated 
was  a  design  study  on  approaches  to  remedying  this  performance  problem.  This  work 
involved  consideration  of  various  asynchronous  and  stream  based  protocols, 
including  possible  use  of  the  existing  ST  protocol  for  high  data  rate  applications. 


A  six  month  extension  and  modification  of  the  work  statement  for  this  contract  was 
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approved  by  the  government  on  26  August  1988.  The  work  statement  was  expanded  to 
provide  for  the  connection  of  the  Jet  Propulsion  Laboratory  (JPL),  located  in  Pasadena, 
CA,  to  the  Wideband  Network  via  the  Wideband  node  at  Information  Sciences  Institute 
(ISI),  located  in  Marina  del  Rey,  CA.  This  connection  is  part  of  a  high-speed 
communication  path  for  the  support  of  distributed  SDI  simulation  work  being  done  at  both 
JPL  and  ESD.  The  simulations  run  on  a  Hypercube  multiprocessor  hardware  base  and  are 
accessible  via  Sun  and  Iris  workstations.  Full  complements  of  all  of  these  machines  are 
currently  installed  on  separate  Ethernet  LANs  at  the  two  sites. 

The  connection  of  JPL  to  the  Wideband  Network  was  completed  in  September.  This  work 
entailed  the  installation  of  a  Butterfly  Internet  Gateway  at  JPL  with  interfaces  to  a  local  JPL 
Ethernet,  and  to  the  remote  BSAT  packet  switch  at  ISI  via  a  terrestrial  T1  circuit.  The 
completion  of  the  JPL-to-ESD  Wideband  path  also  required  the  addition  of  a  new  interface 
to  the  ESD  Butterfly  Internet  Gateway  to  connect  the  local  "ESD-Hypercube"  Ethernet  to 
the  Wideband  Network. 

The  use  of  Cronus  on  the  Wideband  network  was  presented  at  RADC  COTD 
demonstrations  during  December  1988  and  January  1989.  Both  presentations  were 
conducted  at  RADC  using  the  Wideband  Network  to  connect  the  RADC  network  to 
resources  at  BBN.  The  December  demonstration  was  for  RADC  personnel.  The  January 
demonstration  was  part  of  the  RADC  COTD  Technology  Exchange  Meeting,  attended  by 
other  COTD  contractors.  Demonstrations  included  the  Cronus  TVE  simulation  software, 
the  monitoring  and  control  services  (both  Sun-based  Wideband  Network  monitoring 
and  also  distributed  system  service  monitoring  tools  supported  by  Cronus)  and 
distributed  remote  database  services.  Most  all  demonstrations  illustrated  use  of  the 
Wideband  Network  for  resource  sharing  between  RADC  and  BBN. 

The  design  of  a  strategy  for  asynchronous  request  mechanisms  in  Cronus  and  of 
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enhancements  to  their  TCP  connection  management  mechanisms  was  completed  in 
December  1988.  This  work  was  recommended  earlier  by  performance  studies  on  the  TVE 
which  indicated  that  the  long  round-trip  delays  on  the  Wideband  network  would  impact 
performance  of  synchronous  interactions  and  of  certain  uses  of  TCP  connections  common 
to  the  TVE  simulation  software.  The  design  notes  were  referred  to  the  Cronus  group  for 
review.  The  asynchronous  mechanisms,  which  offer  performance  improvements  in  the 
TVE  and  several  other  applications,  regardless  of  network  technology,  were  incorporated 
into  Cronus  and  will  be  supported  in  their  next  release.  The  connection  management 
mechanisms,  which  are  more  complex,  are  planned  for  implementation  for  release  mid 
1989. 

As  a  final  task,  the  Sun-based  Wideband  monitoring  capability  was  carried  further  into  a 
window-based  graphical  system  by  leveraging  off  functionality  already  existing  within  the 
Automated  Network  Management  (ANM)  system.  Using  a  packet  radio  monitoring 
module  as  a  starting  point,  a  prototype  Wideband  Network  monitoring  tool  was  built. 
Complementing  the  Cronus  monitoring  program  which  displays  Cronus  component  status, 
this  Wideband  Network  monitoring  tool  probes  into  the  Wideband  components  for  the 
detailed  status  available  there. 

The  Automated  Network  Management  system  is  a  DARPA-  developed  network 
management  system  based  on  distributed  components  connected  via  a  common 
protocol.  This  commonality  allows  users  to  add  new  components  into  the  system  and 
thereby  take  advantage  of  ANM’s  already  existing  functionality.  Some  features  of  ANM 
that  would  be  of  use  for  monitoring  any  network  are  graphical  color  displays  representing 
network  status  and  the  capability  of  adding  intelligent  trouble  shooting  client  applications 
to  the  system. 

The  goal  of  integrating  the  Wideband  Network  monitoring  into  the  ANM  structure  is  to 
provide  to  the  network  the  capabilities  of  the  ANM  system,  thus  further  enhancing  the 
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management  of  the  Cronus/TVE’s  underlying  networking  substrate.  This  integration  was 
accomplished  by  creating  an  intermediate  process  to  translate  between  the  ANM 
information  format  and  the  format  used  by  the  existing  Wideband  monitor.  The  translator 
receives  reports  from  the  Wideband  monitor  and  sends  repackaged  monitoring  information 
to  the  current  ANM  management  module.  Other  processes,  such  as  the  Integrated 
Monitoring  display  process,  can  then  access  the  management  module  for  status  and 
performance  information  from  the  Wideband  Network.  The  translator  and  Wideband 
monitor  were  not  changed  to  take  advantage  of  all  of  the  possible  ANM  capabilities.  Only 
the  basic  network  status  reports  are  sent  to  the  ANM  management  module.  Also,  an 
already  existing  view  for  a  packet  radio  network  was  used  to  display  the  status  of  the 
Wideband  Network.  The  result,  however,  is  a  graphical  display  which  can  quickly  tell  an 
operator  if  any  Wideband  site  is  not  functioning  properly. 
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4.  CONCLUSIONS 

The  execution  of  the  TVE  simulation  among  the  three  Cronus  clusters  did  put  some  stress 
on  the  Wideband  Packet  Satellite  Network  and  on  the  Butterfly  Internet  Gateways.  The 
Cronus  TCP/IP  traffic  during  long  simulations  involving  several  hosts  helped  uncover 
some  Wideband  problems,  since  it  exercised  the  paths  and  protocols  differently  than,  for 
instance,  the  stream  protocols  used  in  video  conferencing.  The  Wideband  support  group 
analyzed  and  corrected  several  problems  that  a  "real-world"  application  could  help  uncover. 

The  Cronus  project,  with  considerable  experience  in  host-host  communication  on  a  local 
Ethernet,  benefited  from  the  study  of  an  application  run  among  hosts  with  larger 
communication  delays.  Our  study  of  the  Sun  TCP/IP  performance  helped  our 
understanding  of  how  much  can  be  expected  from  vendor-supplied  operating  system 
software  when  used  in  configurations  beyond  a  single  LAN.  The  study  of  the  standard 
Cronus  synchronous  invoke/reply  operation  protocol  has  provoked  a  re-design  of  the  basic 
mechanism.  Future  Cronus  releases  will  have  more  flexible  options,  resulting  in  higher 
effective  bandwidth  for  applications  with  larger  data  flow. 

Both  the  installation  of  the  physical  equipment  responsible  for  the  ESD  Cronus  cluster, 
and  the  on-going  support  were  major  tasks.  In  performing  these  tasks,  both  the  Cronus 
TVE  simulation  users  and  the  network  support  groups  discovered  that  network  monitoring 
functions  needed  improvement  Benefits  are  gained  wherever  a  user  can  both  find  out  if 
all  multi-network  components  are  reachable,  and  can  focus  in  on  the  performance  of  a 
single  network:  piece.  The  monitoring  tools  tested  during  this  contract  should  lead  to  better, 
more  permanent  monitoring  and  control  strategies. 
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